Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants

Review on Chromobacterium Violaceum for Gold Bioleaching from E-waste

AbstractElectronic waste, such as printed circuit boards, are an important secondary resource if processed with environment-friendly technologies for obtaining precious metal, such as gold. The gold bioleaching from electronic waste was recently getting paid attractive attention because its available deposit is limited. This review was focused on Chromobacterium violaceum (C. Violaceum), which was a mesophilic, gram-negative, and facultative anaerobe. C. violaceum has the ability of producing CN− which can dissolve gold from the metallic particles of crushed waste printed circuit boards. This article also provided an overview of cyanide-generation mechanism and the optimal conditions for C. violaceum to achieve maximum amount of cyanide generation. The past achievements and recently scenario of recovery studies carried out on the use of some other microorganisms were compared with C. violaceum. And recently some researchers proposed that combined C. violaceum with chemical methods or other mechanism such as iodide, Pseudomonas aeruginosa and Pseudomonas fluorescens which can reinforce the cyanide generation and improve gold-leaching efficiency. The factors affected the microorganisms on cyanide generation were summarized and the proper conditions were also discussed in this article. And present researches of C. violaceum in gold bioleaching had made good progress which the reported leaching efficiency of gold was over 70%

Constructing two-dimensional molecular networks on metal and semiconducting surfaces : a scanning tunneling microscopy study

The synthesis of two-dimensional organic nanostructures on metal and semiconducting surfaces is studied under ultra-high vacuum (UHV) conditions. Three halogenated organic molecules are investigated using scanning tunneling microscopy (STM) on both metal and semiconducting substrates. The first system studied is the adsorption of brominated tetrathienoanthracene (TBTTA) molecules onto the Si(111) √3×√3 R30°-Ag (Si(111) √3-Ag) surface at room temperature. STM images reveal that at low coverage, the molecules readily migrate to step edges and defects in the √3 overlayer. With increasing coverage, the molecules eventually form compact supramolecular structures. At higher coverage (0.4 - 0.6 monolayers), the spatial extent of the supramolecular structures is often limited by defects in the underlying √3 layer. Our results suggest that the √3-Ag surface provides a relatively inert substrate for the adsorption of TBTTA molecules, and that the supramolecular structures are held together by relatively weak intermolecular forces. The second organic molecule investigated is 2,4,6-tris(4-iodophenyl)-1,3,5-triazine (TIPT). Molecules are deposited onto two related surfaces, Ag(111) and Si(111) √3-Ag. On the Ag(111) surface, TIPT molecules dehalogenate spontaneously upon deposition and form organometallic structures at room temperature. Gentle annealing at ~ 100 °C leads to a more ordered molecular network characterized almost exclusively by hexagons and polymerization was confirmed after further annealing at ~135 °C. On the Si(111) √3-Ag surface TIPT molecules remain largely intact and readily diffuse to step edges and defects in the √3 overlayer. At low coverage, most images display regularly spaced “fuzzy lines” which indicate molecular diffusion at room temperature. At higher coverage (0.4 – 0.8 monolayers), supramolecular domains are formed. The geometry of the cell is similar to an energy optimized 2-d free-standing TIPT layer determined by DFT indicating that de-halogenation does not occur on the Si(111) √3-Ag surface at room temperature and that the supramolecular domains are characterized by zig-zag rows of intact monomers held together primarily by I···H hydrogen-like bonding. Finally, the adsorption of 2,6,10-tribromo-4,8,12-trioxa-3a2-azadibenzo[cd,mn]pyrene (TBTANG) molecules is detailed on both Au(111) and Si(111) √3-Ag surfaces. Dosing TBTANG molecules onto a Au(111) surface at room temperature leads to the self-assembly of intact molecules while deposition onto a hot Au(111) surface yields a complete polymer layer. On the Si(111) √3-Ag surface the molecules display high mobility. With increasing coverage, TBTANG exhibits long-range self-assembly of intact molecules. As the coverage approaches one monolayer, the self-assembled layer extends over the entire surface. Defects in the √3-Ag substrate affect the integrity of domains, but do not limit the size. Preliminary annealing experiments do not lead to polymerization of the TBTANG layer. Rather, annealing at ~ 90℃ leads to disordered regions which nucleate primarily at step edges

Ordered supramolecular oligothiophene structures on passivated silicon surfaces

The adsorption of brominated tetrathienoanthracene molecules (TBTTA) onto the Si(111) 3-Ag surface has been studied. The molecules were absorbed at room temperature and annealed up to 400C. One monolayer of Ag was used to passivate the Si surface. Evidence provided by the scanning tunneling microscope (STM) images indicates that at low coverage the molecules readily form 2-d structures..

ADVANCED POLYMERIC MATERIALS FOR TENDON REPAIR

PhDTendons transfer forces from muscle to bone and allow the locomotion of the body. However, tendons, especially for tendons in the hand, get lacerated commonly in different injuries and the healing of tendon within the narrow channel in the hand will normally lead to tendon adhesion and sacrificed tendon mechanics. Researches have been focused on addressing tendon adhesion prevention but neglecting healed tendon mechanics. This thesis discusses the principles and challenges in the design of biomaterials regarding flexor tendon repair with advanced polymer chemistry and materials science. A rational platform, not only focusing on the prevention of tendon adhesion, but devoting more efforts on final healed properties of tendons via implementing glycopolymer-based materials to guide tendon cells attachment, was designed, fabricated and characterized. Controlled ring opening polymerizations and atom transfer radical polymerizations were combined for the synthesis of miktoarm well-defined block copolymers. Para-fluorine click reactions were then implemented to afford glycopolymers with glucose units. Obtained copolymers were transformed into 3D membranes constituting a porous fibrous structure utilizing electrospinning. The aligned structure was then fabricated to optimize the mechanics of these materials for practical application as well as reconstruct normal tendon physiological structure. Lastly, the toxicity, cell affinity and cell activity of obtained materials were evaluated in vitro employing tendon cells as a cell line to confirm the suitability of obtained platforms for flexor tendon repair.Chinese Scholarship CouncilDonghua Universit

PCDNF: Revisiting Learning-based Point Cloud Denoising via Joint Normal Filtering

Recovering high quality surfaces from noisy point clouds, known as point cloud denoising, is a fundamental yet challenging problem in geometry processing. Most of the existing methods either directly denoise the noisy input or filter raw normals followed by updating point positions. Motivated by the essential interplay between point cloud denoising and normal filtering, we revisit point cloud denoising from a multitask perspective, and propose an end-to-end network, named PCDNF, to denoise point clouds via joint normal filtering. In particular, we introduce an auxiliary normal filtering task to help the overall network remove noise more effectively while preserving geometric features more accurately. In addition to the overall architecture, our network has two novel modules. On one hand, to improve noise removal performance, we design a shape-aware selector to construct the latent tangent space representation of the specific point by comprehensively considering the learned point and normal features and geometry priors. On the other hand, point features are more suitable for describing geometric details, and normal features are more conducive for representing geometric structures (e.g., sharp edges and corners). Combining point and normal features allows us to overcome their weaknesses. Thus, we design a feature refinement module to fuse point and normal features for better recovering geometric information. Extensive evaluations, comparisons, and ablation studies demonstrate that the proposed method outperforms state-of-the-arts for both point cloud denoising and normal filtering

Is ChatGPT a Good Recommender? A Preliminary Study

Recommendation systems have witnessed significant advancements and have been widely used over the past decades. However, most traditional recommendation methods are task-specific and therefore lack efficient generalization ability. Recently, the emergence of ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. Nonetheless, the application of ChatGPT in the recommendation domain has not been thoroughly investigated. In this paper, we employ ChatGPT as a general-purpose recommendation model to explore its potential for transferring extensive linguistic and world knowledge acquired from large-scale corpora to recommendation scenarios. Specifically, we design a set of prompts and evaluate ChatGPT's performance on five recommendation scenarios. Unlike traditional recommendation methods, we do not fine-tune ChatGPT during the entire evaluation process, relying only on the prompts themselves to convert recommendation tasks into natural language tasks. Further, we explore the use of few-shot prompting to inject interaction information that contains user potential interest to help ChatGPT better understand user needs and interests. Comprehensive experimental results on Amazon Beauty dataset show that ChatGPT has achieved promising results in certain tasks and is capable of reaching the baseline level in others. We conduct human evaluations on two explainability-oriented tasks to more accurately evaluate the quality of contents generated by different models. And the human evaluations show ChatGPT can truly understand the provided information and generate clearer and more reasonable results. We hope that our study can inspire researchers to further explore the potential of language models like ChatGPT to improve recommendation performance and contribute to the advancement of the recommendation systems field.Comment: Accepted by CIKM 2023 GenRec Worksho

GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models

Neural architectures can be naturally viewed as computational graphs. Motivated by this perspective, we, in this paper, study neural architecture search (NAS) through the lens of learning random graph models. In contrast to existing NAS methods which largely focus on searching for a single best architecture, i.e, point estimation, we propose GraphPNAS a deep graph generative model that learns a distribution of well-performing architectures. Relying on graph neural networks (GNNs), our GraphPNAS can better capture topologies of good neural architectures and relations between operators therein. Moreover, our graph generator leads to a learnable probabilistic search method that is more flexible and efficient than the commonly used RNN generator and random search methods. Finally, we learn our generator via an efficient reinforcement learning formulation for NAS. To assess the effectiveness of our GraphPNAS, we conduct extensive experiments on three search spaces, including the challenging RandWire on TinyImageNet, ENAS on CIFAR10, and NAS-Bench-101/201. The complexity of RandWire is significantly larger than other search spaces in the literature. We show that our proposed graph generator consistently outperforms RNN-based one and achieves better or comparable performances than state-of-the-art NAS methods